Implementation of a Black-Box Global Optimization Algorithm with a Parallel Branch and Bound Template

A new derivative-free global optimization algorithm is proposed for solving nonlinear global optimization problems. It is based on the Branch and Bound (BnB) algorithm. BnB is a general algorithm to solve optimization problems. Its implementation is done by using the developed template library of BnB algorithms. The robustness of the new algorithm is demonstrated by solving a selection of test problems. We present a short description of our template implementation of the BnB algorithm. A paradigm of domain decomposition (data parallelization) is used to construct a parallel BnB algorithm. MPI is used for underlying communications. To obtain a better load balancing, the BnB template has a load balancing module that allows the redistribution of a search space among the processors at a run time. A parallel version of the user’s algorithm is obtained automatically from a sequential algorithm. 1 Problem Formulation Many problems in engineering, physics, economic and other fields may be formulated as optimization problems, where the minimum/maximum value of an objective function should be found. Branch and bound (BnB) is a general technique to solve optimization problems. It can be used in many optimization algorithms, for example to solve combinatorial optimization or covering global optimization problems. Its general structure can be implemented as an algorithm template that will simplify implementation of specific BnB algorithms to solve a particular problem. Similar template ideas applied for a parallel algorithm relieve users from doing the actual parallel programming. Consider a minimization problem, formulated as f∗ = min X∈D f(X), (1) where f(X) is an objective function, X are decision variables, and D ⊂ R is a search space. Besides of the minimum f∗, one or all minimizers X∗ : f(X∗) = f∗ should be found. The main idea of the BnB algorithm is to detect the subspaces not containing the global minimizers and discard them from the further search. The initial B. K̊agström et al. (Eds.): PARA 2006, LNCS 4699, pp. 1115–1125, 2007. c © Springer-Verlag Berlin Heidelberg 2007 1116 R. Čiegis and M. Baravykaitė search space D is subsequently divided into smaller subspaces Di. Then each subspace is evaluated trying to find out if it can contain the optimal solution. For this purpose a lower bound of the objective function LB(Di) is calculated over the subspace and compared with the upper bound UB(D) for the minimum value. If LB(Di) ≥ UB(D), then the subspace Di cannot contain the global minimizer and therefore it is rejected from the further search. Otherwise it is inserted into the list of unexplored subspaces. The algorithm terminates when there are no subspaces in the list. Unlike the data parallel applications (e.g., algorithms for a solution of partial differential equations) optimization problems are characterized by an unpredictably varying unstructured search space [20]. This property produces additional difficulties for creation of parallel BnB algorithms: a) the change of space search order with respect to the sequential one, b) a load unbalance of processors, c) costs of additional communications. We note that any BnB algorithm depends on few rules, and the optimal selection of these rules is strongly problem dependent. For many engineering applications only values of the objective function f(X) can be computed and we do not have information on the derivatives of f . In this paper, we present a new derivative-free algorithm for a solution of nonlinear global optimization problems. The objective function is computed by a blackbox algorithm. This BnB algorithm is implemented by using a new template library of BnB algorithms. The main goal of this tool is to present a flexible and extendable template library which enables users to make experiments with different strategies of subspace selection, branch and bound rules and techniques for computation of lower bounds of the objective function LB(Di). We also present a short description of the developed template library. A parallel version of user’s algorithm is obtained automatically from the sequential algorithm. Some results of numerical experiments illustrate the efficiency of the template library. The rest of the paper is organized as follows. A generalized BnB algorithm is described in Section 2. A new black-box global optimization algorithm is presented and investigated in Section 3. In Section 4 a template based implementation of the BnB algorithm is considered. Some final conclusions are done in Section 5. 2 A Generalized BnB Algorithm The branch and bound technique is used for managing the list of sub-regions and the process of discarding and partitioning. The general branch and bound algorithm is shown in Figure 1, where L denotes a candidate set, S is the solution, UB(Di) and LB(Di) denote upper and lower bounds for the minimum value of the objective function over sub-space Di. A selection rule, a lower bound computation rule and a branch rule define the given BnB algorithm. There are many different strategies for a selection order of subproblems. The most popular strategies are defined as the best first search, the last first search and the breadth first search rules. A bound rule is Implementation of a Black-Box Global Optimization Algorithm 1117